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ABSTRACT 



In evaluation of communication instruction^ the needs 



of the administrator are no different from the needs of both students 
and facultyt information and feedback freely given and accepte:J is a 
hallmark of an environment that fosters a quality eviucation. Since 
communication instruction is not different from any other kind of 
instruction, its evaluation should not be different either. 
Evaluations that have the potential for generating a quality 
education combine some form of student rating on a form developed at 
least in part by the instructor, given at a point in the semester 
where it will have the greatest benefit, tied to the confidentiality 
of the results, and coupled with significant input from professionals 
and colleagues. All players in this game have rights^ needs, and 
responsibilities. In any communication environment, feedback is 
essential; for the communication to continue, the feedback process 
must be mutual. (RS) 
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WHAT ARE WE COMMUNICATING WHEN WE EVALUATE INSTRUCTION? 
EVALUATION OF COMMUNICATION INSTRUCTION: 
AN ADMINISTRATOR'S PERSPECTIVE 

The issue of the evaluation of communication instruction, is 
really an issue of the viability of academic evaluations in toto, 
learly not restricted to our parochical field(s). Several 
issues are part of this playground, including, though almost 
assuredly not limited to, the following: 

1. Is communication instruction different from any other kind 
of instruction, (thereby needing something in the way of 
evaluation that other fields do not)? 

2. What about the issues of tenure, promotion, merit, retention, 
and initial hiring? 

3. Can evaluation and student rights be talked about in the same 
discussion, without appreciably adding to dissension? What 
is the general mood in country, vis-a-vis academic 
evaluat ion? 

4. As a COMMUNICATION administrator, are the responsibilities, 
or abilities any greater than those of other administra- 
tors, (or any other communication faculty, for that matter?) 

5. What are the academic freedom issues, or stated another 
way, are there academic freedom issues? 

The answer to point one, on the difference between 

communication and other forms of instruction clearly has to be 

no. Whatever differences exist are not pedagogic in nature; we 

teach a discipline, with varying theories, methodologies, styles, 

texts, techniques, et.a. , and that needs to be understood 

whether evaluation is defacto or dejure. 
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Another point needs to be said at this juncture even though 
it is clearly understood to all communication faculty, and 
really, to all in our profession. Evaluation is a communication 
process* It means understanding messages, looking at context and 
content, reacting to all types of stimuli and perhaps most impor- 
tant, giving and receiving feedback. It is perhaps too 
simplistic at the outset to call all instruction evaluation 
feedback, but clearly that is tha route that will be taken here. 
The ultimate goal in a vital institution should not be the 
creation of a paper case for dismissal, but the improvement and 
education of faculty, as well as students. The evaluation 
process, in all its aspects, if properly set up and administered 
will be, ideally, a positive, and only a positive experience. 
The students'' role in the process, that of participating in some 
student-instructor rating system, should not only provide useful 
information, but should also communicate to the student a 
message of excellence in intent, and in operation. 

To understand academic freedom and the related issues, and 
to discuss it requires a definition as a starting point. The 
following, though clearly not universally acceptable does cover 
certain important areas of interest to instructors and 
administrators * 



All members of the faculty, whether tenured 
or not, are entitled to academic freedom as 
set forth in the 1940 Statement of Principles 
on Academic Freedom and Tenure, formulated by 
the Associate of American Colleges and the 
American Association of University Profes- 
sors. 



That statement includes the following: 



The teacher is entitled to freedom in the 
classroom in discussing his subject, but he 
should be careful not to introduce into his 
teaching controversial matter which has no 
relation to his subject. Limitations on 
Academic Freedom should be c 1 ea r 1 v s t a t ed in 
writing at the time of appointment. 



Surely, no one can argue the importance of teaching the subject 
area in an appropriate manner, and it should be acceptable to 
assume that students taking a course listed in a catalog have a 
right to the course content described. The omnipresent litigious 
nature of students deprived of course content at least similar to 
catalog descriptions makes the mandating and enforcing of accep- 
ted (by department) course syllabi a protection not only for the 
institution but for faculty as well. The following proposed 
statement on academic freedom attempts to afford such protection. 



Academic Freedom is the freedom to teach and 
to conduct research in whatever manner a 
professor sees fit, subject to certain 
governmental, legal, and organizational 
constraints. These are as follows: 



1. A professor is obligated to teach the 
contentof a properly approved and published 
course or curriculum, and to meet the stated 
needs of a prerequisite, an accreditation 
requirement, or a contractural commitment. 
The method of teaching ^.s the professor's 
choice. The teaching process shall not, 
however, asssult the dignity or civil rights 
of students, and shall conform to established 
affirmative action and equal opportunity 
regula t ions . 

2. A professor may freely pursue any line of 
inquiry or research and may freely publish his 
or her findings, subject to Federal and State 
regulations, national security constraints, 
corporate contractural clearances, or other 
restrictions properly applicable to the work 
at hand. Professors entering into government 
or corpo ite contracts to perform specific 
research activities are expected to honor 
their commitments and to abide by any publis- 
hing or freedom of in for mat ion constraints. 

3. As an officer of the University, a profes- 
sor is expected to act responsibly and in the 
University's best interests when dealing with 
students, alumni, government agencies, and 
external groups and constituencies. A profes- 
sor should expect that his or her actions may 
at times be taken as representative of Univer- 
sity policy, and that, rightly or wrongly, his 
or her views and opinions may be viewed as 
University position statements.^ 



The answers to the issues affecting tenure, promotion, merit pay 
and raises, and retention can be answered now with a little 
clearer understanding of the scope of the discussion. 
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Formal evaluation of teachers by students, 
something that would have been viewed as the 
height of disrespect a generation ago, has 
become an accepted procedure on most college 
campuses. The evaluations are used to improve 
teaching, to help students choose courses and 
to assist faculty and administrators in 
promotion and tenure decisions. 



Professor John Centra, of Syracuse University 
has said: 



Probably one of the big changes in the last 
decade is the shift from using the information 
strictly as feedback for the course to using 
it extensively for tenure and promotion 
decis ions . 



Thomas C. Greaves, Provost at Bucknell University sees a 
slightly different emphasis, especially from the perspective of 
the tenured faculty. The concept that makes the syclem he 
administers work better is not the fact of evaluation per se, but 
rather the aspect of timing. 



Tenured faculty are evaluated with respect to 
their teaching only once every three years. 



Dr. Greaves sees the potential problems caused by evaluation in 
faculty-administered relationships. No doubt these proble ms are 
heightened when the department chair ia also the evaluator, or at 



least has a direct role In the evaluation process > 



Faculty basically see themselves asself- 
employed within a community of teacher- 
scholars. On the whole, the faculty tend to 
be uncomfortable with an employment model 
which suggests that they work and conform 
their behavior to the desires of 
administration; . . . 



Dr» Greaves identifies the self-seen role of faculty as one of 
fulfillment of career, and mission, i.e. academic, including 
teaching and research. He points out that when faculty see 
career emphasized over mission, as could be the case when 
looking at merit evaluation, stressful situations develop. 
Faculty do recognize the role of student evaluations, however. 



..•faculty generally concede the utility of 
student evaluation of classes and the use of 
the data, in conjunction with other input, in 
discriminating various levels of professional 
success as a teacher* Merit evaluation, 
likewise seers to be a necessary part of that. 
Consequently, merit evaluation poses a 
conflicting dllema for faculty. It is 
stressful, frustrating, and time consuming. 



Greaves ends by commending the Bucknell model, whereby tenured 
faculty are reviewed on a triennial^ rotating basis. That way, 
the "comparative dimensions that justify why person A gets a 
slightly different raise than person B become less salient".^ 



ERIC 



6 

6 



One of the longest running student evaluation of faculty 
programs has existed for forty years at the Unversity of 
Michigan, in Ann Arbor. Dr. W.J. McKeachie, Associate Director 
of the National Center for Research to Improve Po s t s e cond a ry 
Teaching and Learning (NCRIPTAL) at University of Michigan has 
said that although some faculty do not appreciate or utilize the 
evaluations, it has survived in faculty balloting for the forty 
years it has been in existence. He says that supplemental 
methods for evaluation are not uniform from one department to the 
next . 



Some use classroom visitation; some review 
syllabi and examinations; most. ..(in a 
particular College) will have a discussion of 
the individual's teaching among the faculty 
committee that determine salaries and 
promotions. The constitution of these 
committees varies from department to 
department, the committee consists of elected 
members with at least two from each rank, and 
members of the committee do not particpate in 
consideration of their own rank. This 
produces the i^nusual situation that full 
professors are reviewed by associate and 
assistant professors. 



Although not uniformly applied institution - wide, the University 
of Michigan model has some sections that require evidence of 
teaching effectiveness, garnered from student ratings to play a 
role in promotion and merit pay determination. 



One of the leading experts in the discussion of faculty 
evaluation is Dr. Donald X. Smith, Senior Vice-President of the 
University of Wisconsin System. Taking a historical overview of 
the evaluation issue, Smith reviews early personnel issues 
handled predominantly by administration and lay boards and the 
ensuing growth and need based change to a peer review system. 



At least in theory, the department consists of 
a reasonably coherent aggregation of faculty 
members whose professional preparation in 
areas of expertise promote common standards of 
quality and easy communication, whose aree. of 
academic responsibility can therefore be 
def ined . 

The department or equivalent academic unit, 
generates a "community of peers" usually the 
senior or tenured members of the department. 
This community is presumably able to 
evaluate candidates for membership, and 
resources available, to employ new members. 
Such new members usually enter on an 
probationary status, and are evaluated 
annually for salary recommendations, promotion 
recommenations, retention or nonrenewal 
decisions, and ultimately on the decisive 
determination of an sward of tenure or 
separation from the department and university. 



Smith then continues to explain the peer review as one that 

comprises two basic assumptions, those being: 

1. The selections of a member of the faculty 
does not involve find In g someone who can 
simply perform competently the serv- 
ices defined for a given position... and 
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2. ...that judgements ...cannot be derived 
wholly from credent ia Is, point- in -time 
examinations, or even single interviews. 

Further operational assumptions assume that a collegial faculty 
who have the professional and institutional experience over time 
to make such determinations prior to and after the initial 
decision, and that even the initial decision was made with 
n a t iona 1 input of a similar nature. It is an inherent and 
benefically unique attribute of the collegiate system that the 
position description is a variable requiring mediation within the 
noted community of scholars. Smith goes on to point out, 

The faculty personnel system provides security 
slowly and.. .on the basis of a judgement which 
SEEKS MORE THAN COMPETENCE.^ 

A merit system, one in which differentiations are made as part of 
the judgement of the vitality and academic excellence of an 
individual'^c performance are recognized by Smith as a difficult 
task for the groups involved in the decisions affecting 
retention, salary, promotion et.al. However, 

Abandonment of the merit p r in c ip 1 e s , . . . wo u Id 
seem to be a case of faculty members abdica- 
ting a responsibility basic to their claim to 
be officers of their university, primarily 
responsible for its quality.^*' 



Further recognition of the role of mulitple input into the 
advisf.ng system is seen as necessary by Jack Lindquist, Director 
of the Institute for Academic Improvement at Memphis State 



University. He states, 



Student ratings of Instruction, ...as well as 
peer observation of class or critique of 
course syllabi and bibliography, can help 
generate teaching assessment. Such data- 
gathering should be done within the context of 
a relationship with a faculty development 
facilitator, skilled chairperson, trained 
colleague or student who can help interpret 
the data and work out a promising way to 
improve and who has some detachment from the 
situation. 



Lindquist lists ten STRATEGIES FOR IMPROVING COLLEGE TiZACHING. 

Number five on this list is one of the the aspects of the topic 

at hand, namely, student ratings of instruction. Seen as a 

public or private feedback mode, these are intended to provide 

motivation for the rectification of course or teaching problems 

perceived by students. Additional use is seen in this for 

eval'tation of faculty performance, and student selection of 

1 ? 

highly ranked teachers. 

John Centra, mentioned earlier, has researched the 
area of the effectiveness of student ratings for many years 
and has written and collaborated extensively. 
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These assessments are especially benefical as 
estimates of teaching or learning effective- 
ness or as indicators of the academic climate 
in departme its, particularly as it affects 
students. Student assessments do have their 
limitations, however, as indeed do most forms 
of assessments. For this reason, it is 
important that multiple indicators of 
performance and m u 1 1 iple perspectives on 
vitality be employed . 

Centra indentifies certain areas in which he feels students 
assessment and ratings are a unique tool. He maintains that 
students alone can tell how they reacted to an instructor, in 
terms of the instructors' enthusiasm about the subject and about 
teaching, ^nd the instructors ability to stimulate them. The 
students can judge whether the material was presented clearly, 
and if the information was challenging. 

While recognizing that, a significant limitation of student eva- 
luation ss t 1 1 1 is that students are not always the best arbiters 
of academic worth. The student's major liability suggested by 
Centra, is their limited academic wisdom and experiences, and a 
tendency to be shortsighted. These, we can all recognize, are 
the very cha ra ct e r is ts we are trying to address and improve 
during the two, four or more years they sit at our feet, so to 
epeak. With all this recognition in mind however. Centra .Identi- 
fies a handful of major studies cei-ifying the use of student 
ratings in tenure and promotion decisions. This would seem to 




Inalcate that the highly rated instructors get the lioa^s share 
of the students, raises, and promotions. S e r en d ip i t o u s ly , 
students also tend to learn more from the highly rated 
instructors. Centrals prior research also shows. However, there 
is sufficient evidence to suggest that student^s perceptions of 
what they learn affect ratings. The other possible outcomer^ of 
the ratings then would be that an instructor is ranked highly, 
even though the students performed poorer than those taking the 
same course with other instructors, and that highly, or poorer 
rated instructors had students in class who performed and 
received grades along an entire scale. Clearly there was also 
significant variance possible given student perception of 
performance, size of class, time of day, etc. The value then 
lies in the existence of the rating system, and if it is to be 
used for any other pupose than for the s el f -a d 'Ji in is t e red , self- 
evaluated use byan instructor, certain controls need co be in 
place. Centra states 



Student ratings then, can help promote good 
teachers, but they are best used in conjunc- 
tion with otherindicatorsofteaching 
effectiveness such as input from colleagues, 
self-report information, and carefully 
considered estimates of student learning. When 
ratings are used for administrative purposes, 
it is especially important that there be a 
sufficient and representative number of 
students responding for a class, and that the 
rating forms be administered in a prescribed 
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and systematic manner. In addition, global 
ratings, rather than ratings of specific 
practices or behaviors ought to be used, and 
ratings for several courses over a period of 
time would probably provide the best basis for 
making judgements. Certain c o u r s e - r e la t e d 
factors which can affect ratings should also 
be taken into account in interpreting the 
scores. These include class size, (small 
classes. ..get much higher ratings), type of 
course (college required courses are rated 
much lower than electives or major field 
courses), and the subject area of the 
course . 



James A. Kulik would seem to see the need for an additional 
perspective. As Associate Research Scientist, and Associate 
Director of the Center for Research on Learning and Teaching at 
the University of Michigan, (and a colleague of W.J. McKeachie, 
quoted earlier), Kulik is responsible for the development of a 
highly remarkable instructor designed questionaire used in 
student ratings. The system is similar to the Purdue CAFETERIA 
System, developed in the 1970's at the Measurement and Re- 
earch Center at Purdue University. The Purdue system features 
a forty item evaluation form self-constructed by faculty from a 
two hundred item list. Five of the final forty items on the 
final form are university standard. 

The process in place at the CRLT is designed to be totally within 
the instructor's control, in so far as item selection, and survey 
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administration are concernei- The completed forms are returned 
to the Center for processing and the computer results returned to 
the instructor. The CRLT will assist with analysis if requested. 
There is significant motivation for the use of student 
evaluations by instructors at the University of Michigan, though 
not necessarily the method provided by the CRLT. Criteria for 
tenure and promotion in place and enforced (by the College of 
Literature, Science and the Arts), lists the following criteria: 

A. Teaching 

1. List of courses taught 

2. STUDENT AND FACULTY EVALUATIONS AND THE DEPARTMENTAL 
MEAN 

3. Departmental enrichment by candidate 
A. Dissertation service 

5. Candidates statement 

6. Amplified Assessment 

B. Research and Publications 

C. Service 

D* External Evaluation 
F» Additional Comments 



The self published quide to those seeking tenure and/or 

promotion, provided by the Office of the Dean also carries this 

admonition. In regard to the lack of use of teaching evaluations. 

A very few departments arestill not 
undertaking student evaluations of teaching. 
This is inappropriate and only serves to 
diminish the candidate'^s chances for a fair 
evaluat Ion. The Execut ive Committee expects 
that evidence of some evaluation of teaching 
will be provided in all Ins tances . ^ 
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Wllbert J. McKeachie provides the final counterpart to the still 
prevalent comments regarding the use and significance of student 
learning as the final judging factor of a teacher's efforts. He 
recognizes that students who are accoustomed to getting good 
grades will (simply) study harder to compensate for less than 
adequate instruction. That would seem to invalidate the use of 
final exam results as a reliable evaluative measurement of the 
instructor's role in the learning process. McKeachie suggests 
another step. 



Today cognitive and instructional 
psychologists are placing more emphasis upon 
the importance of the way in which knowledge 
is structured as well as upon skills and 
strategies for learning and problem solving. 
Thus our assessment of instruction should look 
to these kinds of outcomes as criteria. 



It is, according to the researcher, difficult to collect all the 

necessary definitive data to perform evaluation. Called for is a 

rather involved set of assessments of student organizational 

techniques, tracking the methods students use to rolate 

important concepts taught. Since it is recognized that this is 

often difficult to obtain, at least consistently, and since the 

fallback would seem to be the student ratings of teaching and 

learning, McKeachie suggests additional data from other sources. 

A cognitive approach to the evaluation of 
instruction also suggests that we look more at 
non-classroom activities of teachers - the 
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ways in which teachers plan student activities 
and assignments to help them develop the kinds 
of learning and thinking strategies that we 
now wish to emphasize. We need to assess the 
degree to which teachers are using appropriate 
resouces, both of technology and experiental 
learning, in achieving these objectives; we 
need to evaluate the degree to which teachers 
help students become aware, in a metii- 
cognitive sense, of their own strategies and 
thinking. Both student reports and teacher 
self-reports could help document such 
activities . ^ ^ 



The goal of teaching being recognized here, that the teacher 

succeeds when the student is equipped to learn more outside the 

classroom than inside offers what is by now apparent, 

...the paradoxical problem that a successful 
teacher may be one who gives students the 
sense that their success in a class is 
dependent upon their own effort and skill 
rather than upon the teacher^s skill. ^ 



Sidebars to the issue should also be considered before the 
'.natter is discussed further in other forums. McKeachie calls our 
attention to the reduced effectiveness of student evaluations 
resulting from overkill. The more an institution requires 
evaluations to be performed, or the greater emphasis that is 
placed on them even "officially , greater will be the students 
exposure and participation. While there can be an increased 
interest for some, others will tend to get tired of filling out 
the forms, and treat them as a chore, rather than a privilege. 
This is especially true if the students perceive that end-of-term 
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evaluations would help here, as would a greater examination of 

current semester materials provided by the instructor, while 

there is still an opportunity for effective feedback. 

Avoidance of negative Impact on the institution is 

imperative. Such an impact would be one in which anxiety and 

distrust are increased, and effectiveness decreased. 

We can do a great deal in affecting the 
teaching of a particular faculty member in a 
particular course, but may have less impact 
upon student^s learning that the total impact 
of a w e 1 1 - c 0 n c e i V e d or i 1 1 - c o n c e i v e d 
curriculum . ^ ^ 



It is also important to keep in mind the fact that academe 

is certainly not alone in its s e If -e va lua t ion mode. The concept 

of the performance-appraisal interview moving consistently up the 

hierarchical ladder, is common. Further, 

• ..most workers do not feel threatened by 
being evaluated. ...one of the greatest 
hungers employees have is to know where they 
stand with management. Researchers have found 
that receiving "persona 1 feedback" correlates 
highly with job satisfaction. Appraisal 
interviews offer a periodic session dedicated 
to providing just that feedback. Furthermore, 
evidence suggests that most workers are 
satisfied with the feedback they receive in 
appraisal interviews.^^ 



So, in conclusion there is really only one tenable posit ion 
on the issue of the evaluation of communication instruct ion. The 
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"needs" of the administrator are no different from the needs of 
both students and faculty; whether represented by external or 
internal union, or under the Yeshiva decision; tenured or 
untenured; senior or junior ranked. A quality education in an 
environment that fosters improvements, vitality and the interests 
of all constituencies is the only possible position to take. 
That quality is one in which information and feedback are freely 
given an^. accepted. The quality is maintained where there is a 
desire to generate the feedback, and to provide what is 
meaningful, helpful and designed to be positive. The intent here 
is to equate feedback with quality. 

The model evaluations - some form of student rating on a 
form developed at least in part by the instructor; given at a 
point in the semester where it will have the greatest benefit; 
tied to the confidentiality of the results; coupled with 
significant other kindsof input from professionals and 
colleagues, has the potential for generating thatquality. 
Clearly, all players in this game have rights, needs and 
responsibilities, and since we are dealing in a society, we 
recognize that the operative word here is people, and not 
something that can or should be totally reduced to numbers in a 
ledger, whether good or bad. In any communication environment, 
feedback is essential; for the communication to continue the 
feedback process must be mutual.* 
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